In human-computer interactions, a computer system needs to dynamically update its visual output presentation when the user asks for a new set of data. The present invention focuses on visual context management, a process that dynamically determines how to incorporate newly requested information into an existing visual context so that users can comprehend all relevant information as a coherent whole. Here, the phrase “visual context” refers to a visual scene that a user perceives when issuing a query. Such a process is often subject to diverse constraints (e.g., ensuring semantic continuity and minimizing visual clutter) and unanticipated information introduced during a human-computer conversation. For example, the introduction of a particular visual object to convey a new piece of data may be affected by the existing visual objects for presenting related pieces of data. Without integrating new information into an existing scene, a user may have difficulty in comparing and combining information.
Since it is very difficult to predict how a human-computer conversation would unfold, it is impractical to plan all possible visual context transformations a priori. Previously, researchers and practitioners have experimented with a greedy or schema-based approach to visual context management. However, these approaches normally handle one constraint at a time and do not consider how the constraints themselves may affect one another. As a result, the visual context management result obtained from such existing approaches may not be desirable.
Accordingly, there is a need for improved visual context management techniques.